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DETAILED ACTION 

1 . The Request for Continued Examination (RCE) filed on 16 May 2005 under 37 CFR 
1.53(d) based on parent Application No. 09/976,188 is acceptable and a RCE has been 
established. An action on the RCE follows. 

2. The amendment filed on 21 March 2005, submitted with the filing of the RCE have been 
received and entered. Claims 1-33 as amended are pending in the application. 

Claim Rejections - 35 USC § 103 
The following is a quotation of 35 U.S. C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

3. Claims 1-4, 6-10, 12-15, 17-21, 23, 25-29 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Partanen et al. U.S. Publication 2003/0001899 (hereinafter "Partanen") and 
Dequan CN 1074542 A (translation of Chinese patent attached). 

Referring to claims 1 and 18, Partanen teaches a method and system comprising user 
interface code evaluating a program field that has focus against information indicative of 
whether the field is configured to receive text input (the interface determines whether the user 
has initiated manuscript input to any point on a touch-activated screen of a display, thereby 
causing a semi-transparent window to be opened) (Partanen: page 1, paragraph 0016); and if the* 
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field is configured to receive text input (if a user activated manuscript input has been received) 
(Partanen: page 1, paragraph 0016), providing a visible user input interface at a displayed 
location relative to the field such that the user input interface is operable to receive handwritten 
data while the field is operable to receive input data (upon user initiation of manuscript input at 
point 36 on the display screen of Figure 1, a semi-transparent window is displayed relative to the 
field, allowing users to input manuscript characters, for example, the letter "I", while the field is 
operable to receive input data such as user input of particular letters and/or numbers from a 
virtual keyboard) (Partanen: page 2, paragraph 0032 and page 3, paragraph 0038); receiving 
handwritten data at the input interface (Partanen: page 2, paragraph 0022 and Figure 1); 
providing the handwritten data to a recognition engine (relaying the pattern of the handwritten 
character to the handwriting recognition software) (Partanen: page 2, paragraph 0025); and 
returning a recognition result to the program (for example, recognizing the handwritten "I" and 
outputting it as the character "I" in Figure 1) (Partanen: page 2, paragraph 0022). However, 
although Partanen teaches a virtual keyboard, Partanen fails to explicitly teach receiving typed 
user input and the program field being operable to receive typed and handwritten user input. 
Dequan teaches an input interface that allows users to enter handwritten data (Dequan: page 5 of 
the attached translation) similar to that of Partanen. In addition, Partanen further teaches 
receiving typed user input and the program field being operable to receive typed and handwritten 
user input (a writing/keyboard dual purpose computer that allows both writing, i.e. handwritten 
and keyboard input) (Dequan: pages 4-5 and Figure 2 of the attached translation). It would have 
been obvious to one of ordinary skill in the art, having the teachings of Partanen and Dequan 
before him at the time the invention was made, to modify the handwriting recognition method of 
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Partanen to include the writing/keyboard capabilities of Dequan. One would have been 
motivated to make such a combination in order to increase the flexibilities and functions of the 
input system, giving the user an option to choose the more convenient input mechanism 
depending on the function to be executed. 

Referring to claims 2 and 19, Partanen, as modified, teach the visible user input interface 
is semi-transparent (Partanen: page 1, paragraphs 0015-0016). 

Referring to claim 3, Partanen, as modified, teach wherein the handwritten data received 
at the input interface is evaluated to determine whether the handwritten data corresponds to a 
gesture (deciphering whether user input is a gesture, such as user selection of virtual buttons and 
keys from a virtual keyboard, or user input of written manuscript characters that requires 
handwriting recognition) (Partanen: pages 2-3, paragraphs 0032-0040). 

Referring to claim 4, Partanen, as modified, teach wherein the handwritten data 
corresponds to a gesture, and further comprising, providing at least one pen event corresponding 
to the gesture to the program (for example, if the handwritten data is a gesture of user selection 
of a virtual button 44 with an external input device such as the pen 30 shown in Figure 2, then 
the pen selection of virtual button 44 causes an event such as deactivation of the semi-transparent - 
window) (Partanen: page 3, paragraphs 0038-0040). 

Referring to claims 6 and 26, Partanen, as modified, teach providing the handwritten data 
to a recognition engine in response to detection of a submit button associated with the visible 
user interface (the semi-transparent user interface provides for the display of a virtual keyboard 
having letters, or buttons, the users can select; furthermore, user handwritten data is recognized 
after the user gesture of touching a field with the stylus to indicate handwriting and causing 
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display of the semi-transparent window) (Partanen: page 1, paragraphs 0014-0016, page 2, 
paragraph 0032 and page 3, paragraphs 0035 and 0038). 

Referring to claim 7, Partanen, as modified, teach providing the handwritten data to a 
recognition engine is performed in response to a time being achieved (displaying the handwritten 
input for a predetermined period of time after the input from the input device, upon which the 
pattern is completed and recognized by the recognition software) (Partanen: page 2, paragraph 
0024 and page 3, paragraphs 0033-0035). 

Referring to claim 8, Partanen, as modified, teach wherein providing the handwritten data 
to a recognition engine is performed in response to a gesture being detected (user handwritten 
data is recognized after the user gesture of touching a field with the stylus to indicate 
handwriting and causing display of the semi-transparent window) (Partanen: page 1, paragraphs 
0014-0016 and page 2, paragraph 0032). 

Referring to claims 9 and 20, Partanen, as modified, teach evaluating at least one window 
attribute corresponding to the field against hard-coded or retrieved information to determine 
whether the field is supported (evaluating, or determining the window attribute of whether the 
user has contacted the displayed window, or screen, with the stylus) (Partanen: page 1, paragraph 
0016 and page 2, paragraph 0032). 

Referring to claim 10, Partanen, as modified, teach accessing window class information 
(accessing window information such as determining whether user has contacted the windowed 
area with a stylus) (Partanen: page 1, paragraph 0016 and page 2, paragraphs 0027-0030 and 
0032). 
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Referring to claim 12, Partanen, as modified, teach adjusting the appearance of the visible 
input window (the semi-transparent window may be moved or sized according to the preferences 
of the user) (Partanen: page 1, paragraph 0015). 

Referring to claim 13, Partanen, as modified, teach increasing the size of the visible input 
window to enable entry of additional handwritten data (increasing the size of the semi- 
transparent window) (Partanen: page 2, paragraph 0031). 

Referring to claim 14, Partanen, as modified, teach erasing the visible input window 
(selection of the virtual button 44 will cause the semi-transparent window to de-activate and 
disappear) (Partanen: page 3, paragraph 0039). 

Referring to claim 15, Partanen, as modified, teach the visible input window is erased in 
response to a close request (selection of the virtual button 44 will cause the semi-transparent 
window to de-activate and disappear) (Partanen: page 3, paragraph 0039). . 

Referring to claim 17, Partanen, as modified, teach wherein the visible input window is 
erased in response to a gesture being detected (upon user gesture of selection of virtual button 
44, the semi-transparent input window is deactivated and disappears) (Partanen: page 3, 
paragraph 0039). 

Referring to claim 21, Partanen, as modified, teach wherein the entered data comprises 
handwritten data (user input of handwritten characters) (Partanen: page 2, paragraph 0022 and 
Figure 1), and further comprising a gesture detection engine that evaluates the handwritten data 
to determine whether the handwritten data corresponds to a gesture (deciphering whether user 
input is a gesture, such as user selection of virtual buttons and keys from a virtual keyboard, or 
user input of written manuscript characters that requires handwriting recognition) (Partanen: 
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pages 2-3, paragraphs 0032-0040), and if so, to provide at least one event to the program (for 
example, if the handwritten data is a gesture of user selection of a virtual button 44 with an 
external input device such as the pen 30 shown in Figure 2, then the pen selection of virtual 
button 44 causes an event such as deactivation of the semi-transparent window) (Partanen: page 
3, paragraphs 0038-0040). 

Referring to claim 23, Partanen, as modified, teach wherein the entered data comprises 
handwritten data (user input of handwritten characters) (Partanen: page 2, paragraph 0022 and 
Figure 1), and further comprising a rulebase that determines an appearance of the visible input 
area including a displayed size thereof (pre-selected preferences that may be established such as 
the display size of the semi-transparent window) (Partanen: page 2, paragraphs 0027-0031). 

Referring to claim 25, Partanen, as modified, teach wherein the visible input area has at 
least one button associated therewith for receiving a command (the semi-transparent window has 
a plurality of buttons that can be selected by the user) (Partanen: page 3, paragraphs 0038-0039 
and further shown in Figure 1). 

Referring to claim 27, Partanen, as modified, teach wherein the user input interface code 
provides the recognition result to the program in a message queue associated with the program 
(as each handwriting character is written, the display signal for displaying the recognized 
handwriting is generated and each characters is displayed on the screen in a queue, i.e. one at a 
time) (Partanen: page 2, paragraphs 0024-0025 and further shown in Figure 1). 

Referring to claim 28, Partanen, as modified, teach wherein the drawing of the visible 
input area positions the visible input area relative to the field based on the information received 
from the field typing engine (the semi-transparent window is displayed at a particular position 
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and in a particular size corresponding to user pre-selected preferences) (Partanen: page 2, 
paragraphs 0027-0031). 

Referring to claim 29, Partanen, as modified, teach wherein the drawing of the visible 
input area sizes the visible input area based on the information received from the field typing 
engine (the semi-transparent window is displayed at a particular position and in a particular size 
corresponding to user pre-selected preferences) (Partanen: page 2, paragraphs 0027-0031). 

4. Claims 5, 16 and 22 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Partanen et al. U.S. Publication 2003/0001899 (hereinafter "Partanen") and Dequan CN 
1074542 A (translation of Chinese patent attached), as applied to claims 1 and 18 above, and 
Frink et al. U.S. Patent 5,956,423 (hereinafter "Frink"). 

Referring to claims 5 and 22, Partanen and Dequan teach all of the limitations as applied 
to claims 1 and 18 above. Specifically, Partanen and Dequan teach a semi-transparent user 
interface and user input gestures (Partanen: page 1, paragraph 0016, page 3, paragraphs 0038- 
0040 and further shown in Figure 2). However, Partanen and Dequan fail to explicitly teach the 
gesture comprising user input directed to an area of the program that is visible through the user 
interface. Frink teaches an interface for inputting and recognizing handwritten data (Frink: 
column 2, line 37-column 3, line 4) similar to that of Partanen and Dequan. In addition, Frink 
further teaches user inputting gestures directed to an area of the program that is visible through 
the interface (Frink: column 2, line 37-column 3, line 4 and column 3, lines 52-64). It would 
have been obvious to one of ordinary skill in the art, having the teachings of Partanen, Dequan 
and Frink before him at the time the invention was made, to modify the handwriting input 



Application/Control Number: 09/976, 188 Page 9 

Art Unit: 2173 

interface comprising gesture detection of Partanen and Dequan to include the input of gestures 
taught by Frink. One would have been motivated to make such a combination in order to allow 
users to easily edit the documents, reducing the confusion of mixing up editing commands and 
data input by the user. 

Referring to claim 16, Partanen, as modified, teach erasing the input window in response 
to a time being achieved (sending the handwritten data to the recognition engine, and therefore 
erasing the input, when the user stops writing for a period of time) (Frink: column 2, lines 44-63 
and Figures 2A, 2B and 2C). 

5. Claim 1 1 is rejected under 35 U.S.C. 103(a) as being unpatentable over Partanen et al. 
U.S. Publication 2003/0001899 (hereinafter "Partanen") and Dequan CN 1074542 A (translation 
of Chinese patent attached), as applied to claim 1 above, and Gilai et al. U.S. Patent 6,018,736 
(hereinafter "Gilai"). 

Referring to claim 1 1, Partanen and Dequan teach all of the limitations as applied to 
claim 1 above. Specifically, Partanen and Dequan teach obtaining information indicative of 
whether the field is configured to receive text input (determining whether the user has initiated 
manuscript input to any point on a touch-activated screen of a display, thereby causing a semi- 
transparent window to be opened) (Partanen: page 1, paragraph 0016). However, Partanen and 
Dequan fail to explicitly teach accessing a database to obtain information. Gilai teaches a 
method comprising handwriting recognition (Gilai: column 23, lines 7-10) similar to that of 
Partanen and Dequan. In addition, Gilai further teaches accessing a database to obtain 
information (Gilai: column 5, line 63 - column 6, line 3). It would have been obvious to one of 
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ordinary skill in the art, having the teachings of Partanen, Dequan and Gilai before him at the 
time the invention was made, to modify the method of determining whether a field is configured 
to receive text input of Partanen and Dequan to include the access of a database to obtain 
information taught by Gilai. One would have been motivated to make such a combination in 
order to provide an organized and efficient way of indexing, storing and retrieving a large 
amount of information. 

6. Claim 24 is rejected under 35 U.S.C. 103(a) as being unpatentable over Partanen et al. 
U.S. Publication 2003/0001899 (hereinafter "Partanen"), Dequan CN 1074542 A (translation of 
Chinese patent attached), as applied to claim 18 above, and Microsoft® Excel, copyright 1999 
(hereinafter "Excel"). 

Referring to claim 24, Partanen and Dequan teach all of the limitations as applied to 
claim 18 above. However, although Partanen and Dequan teach increasing the size of the visible . 
input window (re-sizing the displayed semi-transparent window) (Partanen: page 2, paragraphs 
0027-003 1), Partanen and Dequan fail to explicitly teach increasing the size of the visible input 
window based on the data approaching an end thereof and to enable entry of additional data. 
Excel teaches an input interface that adjusts the appearance of the visible input window (see 
screenshots 2 and 3 attached at the end of the office action) similar to that of Partanen and 
Dequan. In addition, Excel further teaches increasing the size of the visible input window based 
on the data approaching an end thereof and to enable entry of additional data (see screenshots 2 
and 3). It would have been obvious to one of ordinary skill in the art, having the teachings of 
Partanen, Dequan and Excel at the time the invention was made, to modify the input interface of 



Application/Control Number: 09/976, 1 88 Page 1 1 

Art Unit: 2173 

Partanen and Dequan to include increasing the size of the input window as needed, taught by 
Excel. One would have been motivated to make such a combination in order to allow users to 
enter as much information as needed, making it easier for them to input and view information. 

7. Claims 30-33 are rejected under 35 U.S.C. 103(a) as being unpatentable over Partanen et 
al. U.S. Publication 2003/0001899 (hereinafter "Partanen"), Dequan CN 1074542A (translation 
of Chinese patent attached) and Frink et al. U.S. Patent 5,956,423 (hereinafter "Frink"), 

Referring to claim 30, Partanen teaches a system comprising an application program 
having at least one application input area into which user input data can be entered (the email 
program shown in Figure 1 has a plurality of user input areas such as Attachments, Subject, etc., 
into which user data can be entered) (Partanen: page 1, paragraphs 0014-0016); user interface 
code external to the application program (semi-transparent interface window for receiving hand- 
written input) (Partanen: pages 1-2, paragraph 0022, pages 2-3, paragraphs 0032-0040 and 
further shown in Figure 1); a typing engine that determines whether to call the user interface 
code for a selected application input area of the application program based on attribute 
information associated with that application input area (based on attribute information for the 
application input area, such as whether user has initiated manuscript input to a point of the input 
area on a touch-activated screen, it is determined whether the user interface code should be 
called, i.e. whether a semi-transparent window should be displayed) (Partanen: page 1, paragraph 
0016, pages 1-2, paragraph 0022 and pages 2-3, paragraphs 0032-0040), the user interface code 
providing a semi-transparent input area based on the attribute information when called, the semi- 
transparent input area configured such that a user may still enter data via user input into the user 



Application/Control Number: 09/976, 188 Page 12 

Art Unit: 2173 

interface code (upon user initiation of manuscript input at point 36 on the display screen of 
Figure 1, a semi-transparent window is displayed relative to the field, allowing users to input 
manuscript characters, for example, the letter "I", while the field is operable to receive input data 
such as user input of particular letters and/or numbers from a virtual keyboard) (Partanen: page 
2, paragraph 0032 and page 3, paragraph 0038); a gesture engine, the gesture engine invoked to 
determine whether user input data directed to the semi-transparent input area is a gesture directed 
to the application program or information that should be recognized as text (deciphering whether 
user input is a gesture, such as user selection of virtual buttons and keys from a virtual keyboard, 
or user input of written manuscript characters that requires handwriting recognition) (Partanen: 
pages 2-3, paragraphs 0032-0040); and a handwriting recognition engine, the handwriting 
recognition engine configured to receive the information that the gesture engine has decided 
should be recognized as text (when users input handwritten character, the pattern of the 
handwritten is relayed to the handwriting recognition software) (Partanen: page 2, paragraph 
0025), the handwriting recognition engine responding by returning recognized text when 
provided with the information (for example, recognizing the handwritten "I" and outputting it as 
the character "I" in Figure 1) (Partanen: page 2, paragraph 0022). However, although Partanen 
teaches a virtual keyboard, Partanen fails to explicitly teach wherein at least one way in which 
input data may be entered includes user-typed data and the input field being operable to receive 
typed and handwritten user input. Dequan teaches an input interface that allows users to enter 
handwritten data (Dequan: page 5 of the attached translation) similar to that of Partanen. In 
addition, Partanen further teaches wherein at least one way in which input data may be entered 
includes user-typed data and the input field being operable to receive typed and handwritten user 
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input (a writing/keyboard dual purpose computer that allows both writing, i.e. handwritten and 
keyboard input) (Dequan: pages 4-5 and Figure 2 of the attached translation). It would have 
been obvious to one of ordinary skill in the art, having the teachings of Partanen and Dequan 
before him at the time the invention was made, to modify the handwriting recognition method of 
Partanen to include the writing/keyboard capabilities of Dequan. One would have been 
motivated to make such a combination in order to increase the flexibilities and functions of the 
input system, giving user an option to choose the more convenient input mechanism depending 
on the function to be executed. However, although Partanen and Dequan teach a timing 
mechanism (displaying the handwritten input for a predetermined period of time after the input 
from the input device, upon which the pattern is completed and recognized by the recognition 
software) (Partanen: page 2, paragraph 0024 and page 3, paragraphs 0033-0035), Partanen and 
Dequan fail to explicitly teach a timing mechanism configured to cause removal of the semi- 
transparent input area when no user interaction with the visible input area is detected for a period 
of time. Frink teaches an interface for inputting and recognizing handwritten data (Frink: 
column 2, lines 37-65) similar to that of Partanen and Dequan. In addition, Frink further teaches 
a timing mechanism configured to cause removal of the input when no user interaction with the 
input area is detected for a period of time (sending the handwritten data to the recognition 
engine, and therefore erasing the input, when the user stops writing for a period of time) (Frink: 
column 2, lines 44-63 and Figures 2 A, 2B and 2C). It would have been obvious to one of 
ordinary skill in the art, having the teachings of Partanen, Dequan and Frink before him at the 
time the invention was made, to modify the input recognition interface comprising the timing 
mechanism of Partanen and Dequan to include the timing mechanism taught by Frink. One 
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would have been motivated to make such a combination in order to allow users to easily edit the 
documents, reducing the confusion of mixing up editing commands and data input by the user. 
Furthermore, it allows users to perform functions such as note taking faster and more efficiently; 
recognizing characters after a certain time has elapsed, representing the user has completed 
taking notes, is faster and more efficient than translating the written data character by character 
as the user is taking notes. 

Referring to claim 31, Partanen, as modified, teach wherein the recognized text is 
received by the user interface code and made available to the application program (handwritten 
text characters are input through the semi-transparent window and recognized text are displayed 
on the email program shown in Figure 1) (Partanen: page 2, paragraphs 0022-0025 and 0032). 

Referring to claim 32, Partanen, as modified, teach wherein the application program 
displays the recognized text in the application input area (text characters are displayed on the 
semi-transparent window and on area 20 of Figure 1) (Partanen: page 2, paragraphs 0022-0025 
and 0032). 

Referring to claim 33, Partanen, as modified, teach a growth rulebase, the growth 
rulebase determining whether to alter an appearance of the semi-transparent input area in 
response to the information received therein (pre-selected preferences that may be established 
such as the display size of the semi-transparent window can be altered) (Partanen: page 2, 
paragraphs 0027-0031). 



Response to Arguments 
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8. Applicant's arguments with respect to independent claims 1,18 and 30 and corresponding 
dependent claims have been considered but are moot in view of the new ground(s) of rejection. 

9. Applicant's arguments with respect to claim 10 have been fully considered but they are 
not persuasive: The applicant argues that Partanen simply teaches expanding an input field when 
double-tapped and does not show any capability of assessing window attributes or window class 
information. The examiner respectfully disagrees. Partanen teaches that window attributes such 
as whether the user has contacted the displayed window with the stylus, are evaluated in order to 
display a semi-transparent window; furthermore, when it is determined that the user has 
contacted the window, a signal is generated to the data processor to access semi-transparent 
window's attributes, or class information, i.e. the preferences established for opening the semi- 
transparent window, as recited on page 1, paragraph 0016 and page 2, paragraph 0032. 
Therefore, the examiner respectfully maintains that Partanen teaches assessing window attributes 
or window class information. 

10. In addition, the examiner respectfully notes that in the previous office action dated 1 5 
December 2004, the examiner stated that with regards to claim 3, Partanen fails to explicitly 
teach that handwritten data may correspond to a gesture. However, upon closer examiner of the 
Partanen reference, the examiner respectfully asserts that Partanen teaches receiving user input, 
via the stylus, of selection of virtual buttons, as recited on page 3, paragraphs 0038-0040; in 
other words, user selection of virtual buttons with the stylus is a gesture. The examiner 
respectfully points that even the applicant concedes that Partanen comprises teachings of hand 
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gesture recognition, as stated by the applicant on the top of page 14 of the response filed on 21 
March 2005. Therefore, the examiner respectfully asserts that Partanen teaches the input of 
gestures. 



Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Ting Zhou whose telephone number is (571) 272-4058. The 
examiner can normally be reached on Monday - Friday 7:00 am - 4:30 pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, John Cabeca can be reached at (571) 272-4048. The fax phone number for the 
organization where this application or proceeding is assigned is (571) 273-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 
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